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Abstract. Effects of hidden nodes on inference quality of observed network structure 
are explored based on a disordered Ising model with hidden nodes. We first study 
analytically small systems consisting of a few nodes, and find that the magnitude of 
the effective coupling grows as the coupling strength from the hidden common input 
nodes increases, while the field strength of the input node has opposite effects. Insights 
gained from analytic results of small systems are confirmed in numerical simulations 
of large systems. We also find that the inference quality deteriorates as the number 
of hidden nodes increases. Furthermore, increasing field variance of hidden nodes 
improves the inference quality of the effective couplings, but worsens the quality for 
the effective fields. In addition, attenuated coupling strengths involved in at least one 
hidden node lead to high quality of coupling inference. 
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1. Introduction 

Due to recent progresses in multi-electrode recording techniques in experimental 
neuroscience, the neural activity measurement of a population of increasing number 
of neurons becomes possible be], which provides a large opportunity and also a big 
challenge for the large-scale data analysis or dimensionality reduction p], to understand 
how sensory processing, working memory or decision making arises from neural circuits. 
However, the brain region the current recording techniques can measure is very limited, 
therefore, the measured population of neurons is not completely isolated. Still there are 
many unobserved neurons outside this population (some may be upstream neurons), 
but interacting with neurons inside the observed population muum- The inferred 
interactions from the neural data are usually termed functional or effective connectivity, 
since the data fitting captures only part of the statistical features of the spike train data 
for algorithmic simplicity. 

Recently, there appear intensive research interests on the inference problem of a 
random kinetic Ising model with hidden nodes 0 EJ Hoi EQ. However, a systematic 
study of the effects of hidden nodes on the structure inference is still lacking based on 
widely used equilibrium models. Here, we explore these effects based on the maximal 
entropy model (also called the disordered Ising model) extensively used to model and 
fit the neural data in recent studies pana ghe]. We separate an unobserved part 
of network (a hidden subnetwork) from the full model, and probe the effects of hidden 
nodes first in small systems with a few nodes where the analytic study is available, then 
in large systems where we performed extensive numerical simulations under various 
settings. In fact, the correlation observed between two neurons may arise from a hidden 
common input outside the measured population. In addition, the interaction between 
two neurons may be mediated by a chain of unobserved neurons. How the unobserved 
part of the network affects the final inference quality will be the focus of this paper. 

For a model study, the data used for inference are collected with high quality, such 
that the final result does not strongly depend on the inference method. In numerical 
simulations, we used the naive mean-field (nMF) method [15] [L6| T7] to reconstruct 
the interaction strengths between neurons in the observed network. nMF relies simply 
on the inverse of the correlation matrix, which is also suitable for analytic studies of 
small systems with a few nodes. For small systems, we found that the effective field 
of observed neurons receiving a common input increases with the external field (firing 
bias) applied to the hidden input, while neurons interacting directly with the hidden 
input show different behavior from those that do not have a direct interaction. The 
effective couplings between neurons receiving a common input decreases with increasing 
firing bias of the hidden input. However, the neurons without a direct interaction (with 
the hidden input) seem to be unaffected in terms of their inferred coupling values. 
Furthermore, if two neurons are mediated by a chain of hidden neurons, the inferred 
interaction strength decreases with increasing chain length. Insights gained from the 
small size network are confirmed in numerical simulations of large size networks, from 
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which, we showed that the inference performance deteriorates with increasing size of 
the unobserved network, and enhanced coupling strengthes between neurons in the 
unobserved part will increase the inference error for both the couplings and fields. 
Interestingly, if we increase the field strength of the unobserved neurons, the inference 
of couplings for the observed neurons will be improved, while the quality of the held 
inference still deteriorates. 

The rest of this paper is organized as follows. The disordered Ising model 
compatible with the data is defined in Sec. El In Sec. [31 we present an analytic study 
on small systems. In Sec. El we present numerical simulation results on large systems. 
Conclusion is given in Sec. El 

2. Disordered Ising model with hidden nodes 

The experimental data can be described by P independent sampled configurations 
{cr M }(/i = 1,..., P) where a is an N —dimensional vector (the entry of the vector 
takes a binary value ±1) and N is the network size. To build a minimal model to fit 
these data, one can take the constraints up to the second-order correlations in the data, 
resulting in the following maximal entropy model [ 18] : 



( 1 ) 


The coupling terms {J t j} correspond to the correlation constraints (( <Ji<7j) P ^ = 
(<7jC r j) data )) while the held terms {hi} correspond to the magnetization constraints 
(fa) P(cr) = fa) data )- Z is a normalization constant. The symmetric coupling may 
take either positive value or negative value, and the held also has the same situation, 
depending on the data mn- Hence, in general, we call this data-driven model a 
disordered Ising model or a spin glass model [19]. 

From Eq. ([TJ) , one can define an energy term E = — Yhicj hi a ii then the 

distribution in Eq. ([T]) is known as a Boltzmann distribution in statistical mechanics [20]. 
In this paper, we divide the full network into observed (visible) part and unobserved 
(hidden) part, and study the effects of hidden part on the inference quality. We also 
assume the observed neurons have originally no firing biases. Therefore, the energy term 
becomes, 



( 2 ) 


i<j 


where a,b = V, H , and Jb v (JfJ H ) specihes the coupling strength between two neurons 
from the visible (hidden) part, and J}j H ( Jfj v ) indicates the coupling strength between 
two neurons one of which comes from the hidden part. All these coupling strengthes 
follow a Gaussian distribution with zero mean and variance ajfc. The field also follows 
a Gaussian distribution with zero mean and variance ah- In the simulation, one can also 
control the number of neurons in the hidden part, defined by A r h . In the full network, 
each neuron is connected to c other neurons on average without self-interaction. 
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Figure 1. Small systems with hidden nodes. Neurons are indicated by gray circles, 
and the line indicates the direct interaction. The observed part is marked by an ellipse. 
Two neurons receiving a hidden common input (a). Three neurons with only two of 
them receiving a hidden common input (b). The interaction between two neurons 
mediated by a chain of unobserved neurons (c). The dashed line indicates the other 
(L — 3) unobserved neurons. 


In general, it is quite difficult to infer the couplings or fields related to hidden nodes 
given the observed data [2H US!, although some special connection structure could be 
assumed for hidden part to make the inference possible at large network sizes [221 [23]. 
Here, we focus on the effects of hidden part on the inference quality of network structure 
of observed part. For simplicity, we used naive mean-field method to infer the couplings 
and fields in observed part and tested the inference performance against various control 
parameters. First, we define a connected correlation Cij = ( o % a 3 ) data —■ rn % rrij , where the 
magnetization rrii = (0i) data . Then the coupling between neuron i and neuron j can be 
reconstructed by [15] 

■C = -(C-%- (3) 


After the coupling is obtained, the field for neuron i is inferred by na 

i 


hf = tanh (rrii) — y ^ .Jf ' nrij — rrii 


j¥=i 


1 — m- 


-(CT 1 ) 


(4) 


Eq. (E} is derived based on the mean-held equation rn t = tanh( JijfUj + hf) and 
the linear-response theory C t] = dnii/dhj [151 fT6] . A variety of advanced mean held 
approximations can be reduced to this simple approximation under certain conditions, 
e.g., high temperature or small correlation im. The last term in Eq. ([4j) is related to 
an effective self-coupling playing a key role in accurate held inference. However, a more 
natural and accurate way for held inference is applying an adaptive Onsager correction 
term ra- For simplicity, we used the naive mean-held method in this paper. 


3. Small size system: analytic studies 

In this section, we study analytically effects of hidden nodes on small size system, which 
could further provide insights towards understanding these effects on large systems. 
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We first consider a three-neuron system with one common hidden input shown in Fig. [T] 
(a). We assume the observed neurons interact with the hidden neuron with the same 
coupling strength J, and the hidden neuron has a firing bias h. The normalization 
constant Z a = 2 a , cosh( Ja + Jo' + h) where er, o' are states of the observed neurons. 
Therefore, the exact observed connected correlation (■ C ) are given by: 

4 cosh h , 


where m{m r 
is given by: 


cosh(2 J + h) + cosh(2 J — h) + 2 cosh h ’ ^ 

, cosh(2J + h) — cosh(2J — h) . . 

777 = 777 — - ( q ) 

cosh(2 J + h) + cosh(2 J — h) + 2 cosh h' 
is the observed magnetization. According to Eq. (]3]h the inferred coupling 


J =fr 


c 


1 — m 2 


(1 — m 2 )( 1 — m' 2 ) — C 2 
The field can be predicted by: 

hes = K s = tanh _1 (m) - J eS m - m 

1 — m z (1 — m z ) z — C- 
If we increase the field h to a very large value, then we can get the correlation difference 
AC = C(h = 0) — C(h —> oo) = m 2 and the coupling difference AJ eff = 4 , where 

Therefore, both differences are positive, implying that increasing the field 


(7) 


( 8 ) 


m = 


sinh2J 


1+cosh 2 J' 

will lower down both the correlations and the inferred couplings. 

Results are shown in Fig. [2j As observed in Fig. [2] (a), increasing the external 
held of hidden input will lower down the connected correlation between two observed 
neurons. This effect also makes the predicted coupling between two observed neurons 
smaller than that in the presence of a smaller external held, as shown in Fig. [2] (b). 
ffowever, the effective coupling increases as the absolute value of the coupling strength 
(J' H = J) grows. Only when J becomes very weak, J e g- gets close to zero which is the 
true value, due to the nature of the naive mean held approximation. The behavior of 
effective helds with J and h is shown in Fig. [2] (c). The external held of hidden input 
will increase the magnitude of the effective held applied on the observed neurons. For 
large J, the effective held seems to saturate to some large values but smaller than h. 
When J changes its sign, so does h e g, which can be zero only when no hidden input 
interacts with the observed part. 


3.2. Four-neuron system with one common hidden input 

We consider one additional neuron in the observed part, but it doses not interact directly 
with the hidden neuron, as shown in Fig. |T] (b). We assume all J' H = J, J vl = J 
except for the neurons interacting with the hidden neuron ( Jff' = 0). In the observed 
part, three neurons are given the indexes 1, 2, 3. The hidden neuron has a bring bias h. 
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Figure 2. (Color online) Inference results corresponding to Fig. Q](a). (a) connected 
correlation versus coupling strength between observed neurons and the hidden one. 
(b) effective coupling between observed neurons. Note that the true value is zero, (c) 
effective field for observed neurons. Note also that the true value is zero. 


In this case, Zb = 4cosh(2 J)(cosh(2J + h ) + cosh(2J — h)) + 8 cosh h. The connected 
correlation between neuron 1 and neuron 2 is given by: 


C = 


mi = m2 = 


cosh(2J)(cosh(2J + h) + cosh(2J — h)) — 2 cosh h 
cosh(2J)(cosh(2J + h) + cosh(2J — h)) + 2 cosh h 
cosh(2 J)(cosh(2J + h) — cosh(2J — h)) 


- m 1 m 2 , 


cosh( 2 J)(cosh( 2 J + h) + cosh( 2 J — h)) + 2 cosh h 

C i 3 and C 23 are given by: 

sinh(2 J)(cosh(2J + h) + cosh(2J 


C13 — C23 — 


m 3 = 


23 


h)) 


cosh(2J)(cosh(2J + h) + cosh(2J — h)) + 2 cosh h 
sinh(2J)(cosh(2 J + h) — cosh(2J — h)) 
cosh(2J)(cosh(2J + h) + cosh(2J — h)) + 2 cosh h 


mim 3 , 


(9) 

( 10 ) 


( 11 ) 


( 12 ) 
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Let m i = m2 = 7773 = , C\ 3 = C23 = C', we have the following inferred results: 


■A iff 


Jr 


C(1 - m' 2 ) - C’ 2 
A 

C"( 1 — m 2 — C) 
A ’ 


(13) 

(14) 


where J e g and J in are inferred values of J i2 and ^ 23 (^ 13 ) respectively. A = (1 —777 2 )((1 — 
tt7 2 )(1 — m! 2 ) — C' 2 ) — C(C( 1 — m’ 2 ) — C' 2 ) + C 2 (C — 1 + m 2 ). ft is easy to show 
that C{h = 0) — C{h —> 00 ) = (cosh 2 (2J) — 1) cosh 2 (2J)/(l + cosh 2 (2J)) 2 > 0 and 
| C(h = 0)| — | C'(h —> 00 )| = cosh(2J)| sinh(2J)|(cosh 2 (2J) — 1)/(1 + cosh 2 (2J)) 2 > 0. 
One can also show that J e s(h = 0) — J e g(h —> 00 ) = (cosh 4 (2J) — 1)/(8cosh 2 (2J)) > 0 
and Jufh = 0) = J- m (h —> 00 ) = (coslr(2J) + 1) tanh(2J)/4. These results can explain 
the interesting properties shown below. The fields are inferred as: 


h e g = tanh l (m) — J e gm — Ji n m r — m 


h m = tanh 1 (m / ) — 2J m m — m' 


(1 — m 2 )(l — m' 2 ) — C' 2 


1 — m 2 A 

(1 -m 2 ) 2 -C 2 ' 


1 — 777 ' 


a 


A 


(15) 

(16) 


where h e g and h- m are the inferred values of /7i(h 2 ) and h 3 respectively. 

The connected correlation C for neurons 1 and 2 is shown in Fig. [3] (a). Increasing 
the bring bias of the hidden neuron has the effect of lowering the connected correlation, 
which is already observed in a three-neuron system. This also occurs for C (the inset). 
For effective coupling, the behavior becomes rich. First, the firing bias h affects J e g 
in a similar manner to that observed in Fig. [2] (b). However, it does not affect Jj n , 
and J in is close to its true value only when the strength of J is small (| J| < 0.22). As 
observed in Fig. [3] (c), h e g and h in also show different behavior. The magnitude of h in 
is much smaller than that of h e g, whereas, both of them grow with the firing bias h. 
There exists a range of J where the inferred h- m takes a value close to zero. This range 
becomes narrow as h increases. h e ^ as a function of J shows a behavior similar to that 
in the three-neuron system (see Fig. [2] (c)). 


3.3. Two-neuron interaction mediated by a chain of hidden neurons 


The interaction between two observed neurons can also be mediated by a chain of 
unobserved neurons which interact with each other by a coupling strength J (see Fig. [T] 
(c)). Here we assume homogeneous interactions in the hidden part and no firing bias for 
hidden neurons, by focusing on how the effective coupling varies with the chain length 
L and the coupling strength J. This chain will cause a correlation between observed 
neurons pOj . 

C = (tanh J) L . (17) 


And correspondingly, an effective coupling is given by 

(tanh J) L 

eff 1 — (tanhJ) 2L 


( 18 ) 
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Figure 3. (Color online) Inference results corresponding to Fig. |T](b). (a) connected 
correlation versus coupling strength between observed neurons and the hidden one. 
The inset shows the connected correlation C inside the observed network, (b) effective 
coupling between observed neurons. The gray line indicates equality, (c) effective field 
for observed neurons. Note that the true value is zero. The inset shows an enlarged 
view. 


The effect of J and L is shown in Fig. [4] We find that the correlation becomes 
weak as L increases, and the magnitude of the effective coupling also decreases with 
increasing L. J e g is close to the true zero value only when J falls within certain interval, 
otherwise, | «/ e ff I grows with J. This is consistent with the small correlation assumption 
made in the naive mean field approximation [15, 16] . 

4. Large size system: numerical simulations 

For large size system, it is very difficult to perform an analytic calculation. However, 
the main effects of hidden nodes on the inference quality can be probed by numerical 
simulations. Here, we consider a system of size N = 100, and the mean connectivity of 
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(a) 



- 1.0 - 0.5 0.0 0.5 1.0 

(b) J 


Figure 4. (Color online) Inference results corresponding to Fig. |T](c). (a) connected 
correlation versus coupling strength in the hidden part, (b) effective coupling between 
observed neurons. L is the chain length. 



Figure 5. (Color online) (a) Network structure for large size system study. We show 
an example of (N,Nh) = (9,3). The gray circles indicate the hidden nodes while the 
others are visible nodes. In our simulations, we used N = 100 with increasing value of 
Nh- Nodes are sparsely connected with either Poisson or power-law degree distribution. 
The set of hidden nodes is randomly selected for each network instance, (b) The ratio 
between the maximal degree of hidden nodes and that of all nodes. The random 
Poisson graph has mean degree c and the random scale-free graph is characterized by 
the degree distribution P(k) oc fc _7 (fc > The result is averaged over ten random 

instances. 


each node c = 10 unless otherwise specifically stated. Fig. [5] shows the network structure 
we consider. We consider the model defined on random Poisson graphs m first, and 
random scale-free graphs [25j at the end of this section. As the number of hidden 
nodes grows, the nodes of high connectivity (hubs) appear with higher probability in 
the hidden part (see Fig. [5] (b)). These hubs may play an important role in affecting 
the inference quality of the observed part. 



















































Effects of hidden nodes on network structure inference 


10 


The state space of the model is sampled by a standard Monte-Carlo procedure, 
which consists of an asynchronous update of all neurons at an elementary time step (N 
proposed neuron’s state flips), i.e., the state of neuron i , say ex* is updated by 

Prob(<Ji ->■ -cq) = exp (-2cqF/j), (19) 


where Hi = hi + and the update goes over all i. The experimental data 

is collected as P = 10 5 independent configurations (the inference will become more 
accurate with larger P [26]): each of them was sampled with an interval equal to 40 
elementary time steps after sufficient thermal equilibration. These data are used to 
compute correlations and magnetizations. The inference quality is evaluated by the 
(relative) root-mean-square errors: 


h = 

Ah = 




1 1/2 


V 




E M_ - Wf 

N-N h 


1/2 


( 20 ) 

( 21 ) 


Jfy (hfi) represents the true value. The reported results are the average over ten random 
realizations of the network, with the error bars showing the standard deviation. 


f.l. Inference performance on random Poisson graph 

4-1.1. Inference performance with increasing coupling variance Fig. [6] reports results 
on networks with hf = 0 for any hidden neuron i. Increasing the number of unobserved 
neurons, one observes that the inference error Sj also increases. One possible reason 
is, the growing unobserved part yields larger and larger correlations among observed 
neurons, resembling a glassy effect caused by increasing the coupling variance id- 
These correlations may contain higher-order ones. The overall effect is to cause some 
predicted couplings deviate strongly from their true values. The error increases with <jj 
as well, which is consistent with findings obtained in small systems (see Fig. [2] (b) and 

Fig. 0(b)). 

As shown in Fig. [7J by increasing c but maintaining the same value of crj/c , one 
should observe a larger effect of the hidden nodes, since this will increase the probability 
for a hidden node to have an interaction with the observed nodes. However, if only oj is 
fixed, the error will decrease with c because the overall strength of coupling is weakened. 


4.1.2. Inference performance with increasing field variance of hidden nodes To explore 
the effect of the field strength of hidden nodes, one can keep a small coupling variance 
while increasing the value of cr fe . Interestingly, the coupling error decreases as cr^ 
increases, which becomes more apparent when Nh gets larger, as shown in Fig. [E] 
(a). This result is consistent with that observed in Fig. [2] (b) and Fig. [3] (b). This 
can be explained by the fact that, increasing the number of unobserved neurons will 
make more observed neurons interact directly with the hidden ones, while the effective 








Effects of hidden nodes on network structure inference 


11 



Figure 6. (Color online) Inference performance on random Poisson graphs (c = 10) 
without firing biases for hidden neurons. 



Figure 7. (Color online) Inference performance on random Poisson graphs versus c 
without firing biases for hidden neurons. 


couplings between these observed neurons are expected to give a large contribution to 
the inference error. In particular, increasing field strength of hidden nodes results in 
smaller correlations, and thus the coupling prediction can be improved. This point can 
be easily understood from the analytic study of small systems, as shown in Fig. [2] (a) 
and Fig. [3] (a). Compatible with this effect, we also show the global mean correlation 


defined as C — 
nodes, in the inse 


■E 


a 2 

(i,j)ev obs h 


1/2 


where V Q hs denotes the pair set of observed 
We see clearly the global correlation decreases as the 


|Pobs | 

of Fig. m 
held variance increases. 

In contrast to the coupling error, the held error still grows with the held variance, 
which becomes more evident at larger Nh, as shown in Fig. [8] (b). This may be related 
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to the fact that, the effective fields of observed neurons interacting directly with the 
hidden neurons yield a larger contribution to the inference error, compared to those 
inside the observed part (not on the boundary between observed and unobserved part). 
The effective fields of the boundary neurons seem to be very sensitive to changes of 
firing biases of hidden neurons, as observed in Fig. [2] (c) and Fig. [3] (c). 

As shown in Fig. [8] (c), in the presence of hidden nodes, the inferred coupling 
values over-estimate the true large positive couplings, while the large (in absolute value) 
negative couplings are slightly under-estimated, which was also observed in similar 
works [271 [28]. As Nh varies, the inference error is mainly caused by non-existent 
connections and those connections with weak couplings. 

4-1.3. Inference performance with re-scaled couplings Keeping a low value of ah, one 
can also explore the effect of coupling strength among hidden neurons or between hidden 
and visible neurons, by re-scaling the couplings, i.e., all J VH and J HH (hidden couplings) 
are enhanced by a factor J —* gj or attenuated as J —* J/g, where we choose g — 2. 
Results are reported in Fig. El This case corresponds to adding a large perturbation 
to the couplings related to the hidden neurons, with the consequence that both the 
coupling and the field error increase when hidden couplings are enhanced. This finding 
is also consistent with the results reported in small systems (see Fig. [2] and Fig. [3]). 

4-2. Inference performance on random scale-free graph 

In this section, we study the effects of hidden nodes on random scale-free graph. Here the 
coupling strength follows the binary distribution p(J) = pd(J — Jo) + (1 —rj)5(J + Jo) and 
the field is kept constant ( h = h 0 ). In this case, p = 1.0 corresponds to the ferromagnetic 
model while p = 0.0 corresponds to the anti-ferromagnetic model. The behavior Fig. ITOl 
(a) shows is similar to that observed in Fig. [6] and Fig. [8] Note that when Nh becomes 
small, the relative error 5j shows larger value at smaller Jo. This is because the overall 
strength of the denominator in the definition of 5j dominates the error when Nh is 
small, and does not mean that the inference quality at the large Jo is better than that 
at the small Jo (see Fig. [TO] (b) for the scatter plots of a typical example). Fig. [TO] 
(c) shows the inference performance of ferromagnetic and anti-ferromagnetic models, 
implying that the inference quality of either coupling or field deteriorates as the number 
of hidden nodes increases. 

5. Conclusion 

In this work, we study effects of unobserved part of a network on the inference quality of 
network structure of the observed part. We first study analytically the small size network 
with a few neurons, and find that, the effective couplings between the boundary neurons 
decreases as the bring bias of the hidden common input neuron increases, whereas, this 
firing bias does not affect the effective couplings inside the observed network. These 
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Figure 8. (Color online) Inference performance on random Poisson graphs (c = 10) 
with tjj = 0.2. (a) Inference error for couplings versus the number of hidden neurons. 
The inset gives the global correlation versus the number of hidden neurons (the error 
bars are smaller than the symbol size), (b) Inference error for fields versus the number 
of hidden neurons, (c) The scatter plot comparing inferred couplings with the true 
ones for a typical example with <7h = 0.15. The line indicates equality. The inset 
shows the scatter plot for fields. 


effective couplings grow as the magnitude of the input coupling strengthes increases. 
Increasing bring bias of the hidden input will also increase the effective held of the 
boundary neurons which show different behavior from those neurons inside the observed 
part. 

All these interesting properties are also observed in numerical simulations of large 
networks. The inference quality for both couplings and fields deteriorates as the size 
of the unobserved part grows, which can be explained by the fact that an increasing 
number of hidden neurons causes higher-order correlations in the observed data and 
furthermore a larger deviation of the inferred coupling from its true value, mimicking 
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(a) 


(b) 



Figure 9. (Color online) Inference performance on random Poisson graphs (c = 10) 
with <jj = 0.2 and crh = 0.01. The interaction involved in at least one of the hidden 
neurons is enhanced by a factor g = 2 or attenuated by a factor 1 /g. (a) inference 
error for couplings versus the number of hidden neurons, (b) inference error for fields 
versus the number of hidden neurons. 


a glassy phase arising in observed networks. Interestingly, increasing field variance of 
hidden neurons improves the inference quality of the effective couplings, but worsens 
the quality for the effective fields. In addition, attenuated coupling strengthes involved 
in at least one hidden neuron lead to high quality of coupling inference. Our work 
demonstrates the hidden part in a full network does have a significant influence on the 
inference quality of the observed network structure (for both Poisson graph and scale- 
free graph), showing many interesting properties, as revealed in both small networks 
and large networks. 

As new advanced experimental recording techniques are proposed, the number of 
measured neurons becomes larger and larger, providing greater challenges for large-scale 
neural data analysis. The effects of hidden neurons on the inference quality, based on a 
toy model study in this paper, provide insights towards understanding the interaction 
between observed part and unobserved part in terms of network structure prediction. 
Another interesting extension of the current work is to consider the Potts model which 
is widely used in protein structure prediction [29] (and references therein). 
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Figure 10. (Color online) Inference performance on random scale-free graph (fc m ; n = 
5,7 = 3). (a) Inference error versus the number of hidden neurons with rj = 0.5. 
(b) The scatter plot for a typical example with 77 = 0.5 and ho = 0.15. The line 
indicates equality, (c) Inference error for ferromagnetic and anti-ferromagnetic model 
with Jq = 0.2 and Hq = 0.15. 
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